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Abstract 

One of the obstacles in automatic program proving is to obtain suit- 
able loop invariants. The invariant of a loop is a weakened form of its 
postcondition (the loop's goal, also known as its contract); the present 
work takes advantage of this observation by using the postcondition as 
the basis for invariant inference, using various heuristics such as "uncou- 
pling" which prove useful in many important algorithms. Thanks to these 
heuristics, the technique is able to infer invariants for a large variety of 
loop examples. We present the theory behind the technique, its implemen- 
tation (freely available for download and currently relying on Microsoft 
Research's Boogie tool), and the results obtained. 

1 Overview 

Many of the important contributions to the advancement of program proving 
have been, rather than grand new concepts, specific developments and simplifi- 
cations; they have removed one obstacle after another preventing the large-scale 
application of proof techniques to realistic programs built by ordinary program- 
mers in ordinary projects. The work described here seeks to achieve such a 
practical advance by automatically generating an essential ingredient of proof 
techniques: loop invariants. The key idea is that invariant generation should use 
not just the text of a loop but its postcondition. Using this insight, the gin-pink 
tool presented here is able to infer loop invariants for non-trivial algorithms in- 
cluding array partitioning (for Quicksort), sequential search, coincidence count, 
and many others. The tool is available for free download^ 

1.1 Taking advantage of postconditions 

In the standard Floyd-Hoare approach to program proving, loop invariants are 
arguably the biggest practical obstacle to full automation of the proof process. 
Given a routine's specification (contract), in particular its postcondition, the 
proof process consists of deriving intermediate properties, or verification con- 
ditions, at every point in the program text. Straightforward techniques yield 
verification conditions for basic instructions, such as assignment, and basic con- 
trol structures, such as sequence and conditional. The main difficulty is the 
loop control structure, where the needed verification condition is a loop invari- 
ant, which unlike the other cases cannot be computed through simple rules; 
finding the appropriate loop invariant usually requires human invention. 

Experience shows, however, that many programmers find it hard to come 
up with invariants. This raises the question of devising automatic techniques 
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to infer invariants from the loop's text, in effect extending to loops the mech- 
anisms that successfully compute verification conditions for other constructs. 
Loops, however, are intrinsically more difficult constructs than (for example) 
assignments and conditionals, so that in the current state of the art we can only 
hope for heuristics applicable to specific cases, rather than general algorithms 
guaranteed to yield a correct result in all cases. 

While there has been considerable research on loop invariant generation and 
many interesting results (reviewed in the literature survey of Section ^ , most 
existing approaches are constrained by a fundamental limitation: to obtain 
the invariant they only consider the implementation of a loop. In addition to 
raising epistemological problems explained next, such techniques can only try 
to discover relationships between successive loop iterations; this prevents them 
from discovering many important classes of invariants. 

The distinctive feature of the present work is that it uses the postcondition 
of a loop for inferring its invariant. The postcondition is a higher-level view of 
the loop, describing its goal, and hence allows inferring the correct invariant in 
many more cases. As will be explained in Section IXTl this result follows from 
the observation that a loop invariant is always a weakened form of the loop's 
postcondition. Invariant inference as achieved in the present work then relies on 
implementing a number of heuristics for weakening postconditions into invari- 
ants; Section [2] presents four such heuristics, such as uncoupling and constant 
relaxation, which turn out to cover many practical cases. 

1.2 Inferring assertions: the Assertion Inference Paradox 

Any program-proving technique that attempts to infer specification elements 
(such as loop invariants) from program texts faces a serious epistemological 
objection, which we may call the Assertion Inference Paradox. 

The Assertion Inference Paradox is a risk of vicious circle. The goal of 
program proving is to establish program correctness. A program is correct if 
its implementation satisfies its specification; for example a square root routine 
implements a certain algorithm, intended to reach a final state satisfying the 
specification that the square of the result is, within numerical tolerance, equal 
to the input. To talk about correctness requires having both elements, the 
implementation and the specification, and assessing one against the other. But if 
we infer the specification from the implementation, does the exercise not become 
vacuous? Surely, the proof will succeed, but it will not teach us anything since 
it loses the fundamental property of independence between the mathematical 
property to be achieved and the software artifact that attempts to achieve it — 
the problem and the solution. 

To mitigate the Assertion Inference Paradox objection, one may invoke the 
following arguments: 

• The Paradox only arises if the goal is to prove correctness. Specification 
inference can have other applications, such as reverse-engineering legacy 
software. 

• Another possible goal of inferring a specification may be to present it 
to a programmer, who will examine it for consistency with an intuitive 
understanding of its intended behavior. 
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• Specification inference may produce an inconsistent specification, reveal- 
ing a flaw in the implementation. 

For applications to program proving, however, the contradiction remains; an 
inferred specification not exhibiting any inconsistencies cannot provide a sound 
basis for a proof process. 

For that reason, the present work refrains from attempting specification in- 
ference for the principal units of a software system: routines (functions, meth- 
ods) and those at an even higher level of granularity (such as classes). It assumes 
that these routine specifications are available. Most likely they will have been 
written explicitly by humans, although their origin does not matter for the rest 
of the discussion. 

What docs matter is that once we have routine specifications, it becomes 
desirable to infer the specifications of all lower-level constructs (elementary in- 
structions and control structures such as conditionals and loops) automatically. 
At those lower levels, the methodological objection expressed by the Assertion 
Inference Paradox vanishes: the specifications are only useful to express the 
semantics of implementation constructs, not to guess the software's intent. The 
task then becomes: given a routine specification — typically, a precondition and 
postcondition — derive the proof automatically by inferring verification condi- 
tions for the constructs used in the routine and proving that the constructs 
satisfy these conditions. No vicious circle is created. 

For basic constructs such as assignments and conditional instructions, the 
machinery of Floyd-Hoare logic makes this task straightforward. The principal 
remaining difficulty is for loops, since the approach requires exhibiting a loop 
invariant, also known as an inductive assertion, and proving that the loop's 
initialization establishes the invariant and that every execution of the body 
(when the exit condition is not satisfied) preserves it. 

A loop invariant captures the essence of the loop. Methodologically, it is 
desirable that programmers devise the invariant while or before devising the 
loop. As noted, however, many programmers have difficulty coming up with loop 
invariants. This makes invariants an attractive target for automatic inference. 

In the present work, then, postconditions are known and loop invariants 
inferred. The approach has two complementary benefits: 

• It does not raise the risk of circular reasoning since the specification of 
every program unit is given from the outside, not inferred. 

• Having this specification of a loop's context available gives a considerable 
boost to loop invariant inference techniques. While there is a considerable 
literature on invariant inference, it is surprising that none of the references 
with which we are familiar use postconditions. Taking advantage of post- 
conditions makes it possible — as described in the rest of this paper — 
to derive the invariants of many important and sometimes sophisticated 
loop algorithms that had so far eluded other techniques. 

2 Illustrative examples 

This section presents the fundamental ideas behind the loop-invariant generation 
technique detailed in Section 2] and demonstrates them on a few examples. It 
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uses an EifFel-like [3D] pseudocode, which facihtates the presentation thanks to 
the native syntax for contracts and loop invariants. 

As aheady previewed, the core idea is to generate candidate invariants by 
weakening postconditions according to a few commonly recurring patterns. The 
patterns capture some basic ways in which loop iterations modify the program 
state towards achieving the postcondition. Drawing both from classic litera- 
ture [191 129) and our own more recent investigations we consider the following 
fundamental patterns. 

Constant relaxation [291 119] : replace one or more constants by variables. 

Uncoupling [29j : replace two occurrences of the same variable each by a dif- 
ferent variable. 

Term dropping [19]: remove a term, usually a conjunct. 

Variable aging: replace a variable by an expression that represents the value 
the variable had at previous iterations of the loop. 

These patterns are then usually used in combination, yielding a number of 
weakened postconditions. Each of these candidate invariants is then tested 
for initiation and consecution (see Section 13. ip over any loop, and all verified 
invariants are retained. 

The following examples show each of these patterns in action. The tool 
described in Sections E] and [S] can correctly infer invariants of these (and more 
complex) examples. 

2.1 Constant relaxation 

Consider the following routine to compute the maximum value in an array. 
1 max {A: ARRAY [T]; n: INTEGER): T 



2 require A. length = n >1 

3 local i: INTEGER 

4 do 

5 from i := 0; Result := ^[1]; 

6 until i >n 

7 loop 

8 i -.^ i + 1 

9 if Result <^[i] then Result := A[i] end 

10 end 

11 ensure Vj • 1 < j A j < n => A[j] < Result 



Lines 5-10 may modify variables i and Result but they do not affect in- 
put argument ra, which is therefore a constant with respect to the loop body. 
The constant relaxation technique replaces every occurrence of the constant n 
by a variable i. The weakened postcondition, V j • 1 < j A j < i =^ A[j] < 
Result, is indeed an invariant of the loop: after every iteration the value of 
Result is the maximum value of array A over range 



4 



2.2 Variable aging 

Sometimes substituting a constant by a variable in the postcondition does not 
yield any loop invariant because of how the loop body updates the variable. It 
may happen that the loop body does not "use" the latest value of the substituted 
variable until the next iteration. Consider for example another implementation 
of computing the maximum of an array, which increments variable i after using 
it, so that only the range — 1] of array A has been inspected after every 
iteration. 

1 max.v2 {A: ARRAY [T], n: INTEGER): T 



2 require A. length = n >1 

3 local i: INTEGER 

4 do 

5 from i := 1; Result A[l]; 

6 until i >n 

7 loop 

8 if Result <^[i] then Result := A[i] end 

9 i ■= i + I 

10 end 

11 ensure V j • 1 < j A j < n A[j] < Result 



The variable aging heuristics handles these cases by introducing an expression 
that represents the value of the variable at the previous iteration in terms of 
its current value. In the case of routine max-v2 it is straightforward that such 
an expression for variable i is i—1. The postcondition can be weakened by first 
replacing variable n by variable i and then by "aging" variable i into i—1. The 
resulting formula Vj»l<jA j < i—1 =^A[j] < Result correctly captures 
the semantics of the loop. 

Computing the symbolic value of a variable at the "previous" iteration can 
be quite complex in the general case. In practice, however, a simple (e.g., 
flow-insensitive) approximation is often enough to get significant results. The 
experiments of Section [5] provide a partial evidence to support this conjecture. 

2.3 Uncoupling 

Consider the task (used as part of the Quicksort algorithm) of partitioning an 
array A of length n into two parts such that every element of the first part is 
less than or equal to a given pivot value and every element of the second part 
is greater than or equal to it. The following contracted routine specifies and 
implements such task. 

1 partition {A: ARRAY [T]; n: INTEGER; pivot: T): INTEGER 



2 require A. length ~ n >1 

3 local low_index, high_index: INTEGER 

4 do 

5 from low-index :— 1; high_index := n 

6 until low_index — high_index 

7 loop 

8 from no loop initialization 

9 until low_index = high_index V A[low_index] > pivot 
10 loop low-index := low-index + 1 end 
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11 from no loop initialization 

12 until lowJndex = high-index V pivot> A[high-index] 

13 loop high-index := high-index — 1 end 

14 A. swap {A, low-index, high-index) 

15 end 

16 if pivot < A[low-index] then 

17 low-index low-index — 1 

18 high-index := low-index 

19 end 

20 Result := low-index 

21 ensure (Vfc • 1 < k Ak < Result + 1 < pivot ) 

22 A ( V fc • Result <kAk<n^ A[k] > pivot ) 

The postcondition consists of the conjunction of two formulas (lines 21 and 
22). If wc try to weaken it by replacing constant Result by variable low-index 
or by variable high-index we obtain no valid loop invariant. This is because 
the two clauses of the postcondition should refer, respectively, to portion [1.. 
low -index— 1] and [high-index+l..n] of the array. We achieve this by first uncou- 
pling Result, which means replacing its first occurrence (in line 21) by variable 
low-index and its second occurrence (in line 22) by variable high-index. After 
"aging" variable low-index we get the formula: 

(Vfc • 1 < k A k < low-index =^ A[k\ < pivot ) 
A ( V • high-index <k A k < n A[k] > pivot ) 

The reader can check that this indeed a loop invariant of all loops in routine 
partition and that it allows a straightforward partial correctness proof of the 
implementation. 

2.4 Term dropping 

The last weakening pattern that we consider consists simply of removing a part 
of the postcondition. The formula to be weakened is usually assumed to be in 
conjunctive normal form, that is, expressed as the conjunction of a few clauses: 
then term dropping amounts to removing one or more conjuncts. Going back 
to the example of partition, let us drop the first conjunct in the postcondition. 
The resulting formula 

V • Result < A k<n =J> A[k] > pivot 

can be further transformed through constant relaxation, so that we end up with 
a conjunct of the invariant previously obtained by uncoupling: \/ k • high-index 
<k A k< n =^ A[k]> pivot. This conjunct is also by itself an invariant. In 
this example term dropping achieved by different means the same result as 
uncoupling. 

3 Foundations 

Having seen typical examples we now look at the technical choices that support 
the invariant inference tools. To decouple the loop-invariant generation tech- 
nique as much as possible from the specifics of any one programming language. 
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we adopt Boogie from Microsoft Research as our concrete programming lan- 
guage; Section [221 is then devoted to a concise introduction to the features of 
Boogie that are essential for the remainder. Sections 13.11 and [ 3.31 introduce defi- 
nitions of basic concepts and some notational conventions that will be used. We 
assume the reader is familiar with standard formal definitions of the axiomatic 
semantics of imperative programs. 

3.1 Invariants 

Proving a procedure correct amounts to verifying that: 

1. Every computation terminates. 

2. Every call of another procedure is issued only when the preconditions of 
the callee hold. 

3. The postconditions hold upon termination. 

It is impossible to establish these facts automatically for all programs but 
the most trivial ones without additional information provided by the user in the 
form of annotations. The most crucial aspect is the characterization of loops, 
where the expressive power of universal computation lies. A standard technique 
to abstract the semantics of any number of iterations of a loop is by means of 
loop invariants. 

Definition 1 (Inductive loop invariant). Formula (j) is an inductive invariant of 
loop 

from Init until Exit loop Body end 

iff: 

• Initiation: (j) holds after the execution of Init 

• Consecution: the truth of (j) is preserved by every execution of Body where 
Exit does not hold 

In the rest of the discussion, inductive invariants will be called just invariants 
for short. Note, however, that an invariant in the weaker sense of a property 
that stays true throughout the loop's execution is not necessarily an inductive 
invariant: in 

from a; := 1 until False loop x :~ — x end 

formula a; > — 1 will remain true throughout, but is not considered an inductive 
invariant because {x > —1} x := —x {x > —1} is not a correct Hoare triple. In 
the remainder we will deal solely with inductive loop invariants, as is customary 
in the program proving literature. 

From a design methodology perspective, the invariant expresses a weakened 
form of the postcondition. More precisely [22 [H] , the invariant is a form of the 
postcondition that applies to a subset of the data, and satisfies the following 
three properties: 

1. It is strong enough to yield the postcondition when combined with the 
exit condition (which states that the loop has covered the entire data). 



7 



2. It is weak enough to make it easy to write an algorithm (the loop initial- 
ization Init) that will satisfy the invariant on a subset (usually empty or 
trivial) of the data. 

3. It is weak enough to make it easy to write an algorithm (the loop body 
Body) that, given that the invariant holds on a subset of the data that is 
not the entire data, extends it to cover a slightly larger subset. 

"Easy" , in the last two conditions, means "much easier than solving the 
entire original problem" . The loop consists of an approximation strategy that 
starts with the initialization, establishing the invariant, then retains the invari- 
ant while extending the scope by successive approximations to an ever larger 
set of the input through repeated executions of the loop body, until it hits the 
exit condition, signaling that it now covers the entire data and hence satis- 
fies the postcondition. This explains that the various strategies of Section [21 
such as constant relaxation and uncoupling, are heuristics for weakening the 
postcondition. 

3.2 Boogie 

Boogie, now in its second version, is both an intermediate verification language 
and a verification tool. 

The Boogie language combines a typed logical specification language with 
an in-the-small imperative programming language with variables, procedures, 
contracts, and annotations. The type system comprises a few basic primitive 
types as well as type constructors such as one- and two-dimensional arrays. 
It supports a relatively straightforward encoding of object-oriented language 
constructs. Indeed, Boogie is part of the Spec^^ programming environment; 
mappings have been defined for other programming languages, including Eiffel 
PP] and C . This suggests that the results described here can be generalized 
to many other contexts. 

The Boogie tool verifies conformance of a procedure to its specification by 
generating verification conditions (VC) and feeding them to an automated theo- 
rem prover (the standard one being Z3). The outcome of a verification attempt 
can be successful or unsuccessful. In the latter case the tool provides some feed- 
back on what might be wrong in the procedure, in particular by pointing out 
what contracts or annotations it could not verify. Verification with Boogie is 
sound but incomplete: a verified procedure is always guaranteed to be correct, 
while an unsuccessful verification attempt might simply be due to limitations 
of the technology. 

3.2.1 The Boogie specification language 

The Boogie specification language is essentially a typed predicate calculus with 
equality and arithmetic. Correspondingly, formulas — that is, logic expres- 
sions — are built by combining atomic constants, logic variables, and program 
variables with relational and arithmetic operators, as well as with Boolean con- 
nectives and quantifiers. For example, the following formula (from Section [2.ip 
states that no element in array X within positions 1 and n is larger than v: in 
other words, the array has maximum v. 



V j : int • 1 < j A j < n 
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The syntactic classes Id, Number, and Map represent constant and variable 
identifiers, numbers, and mappings, respectively. 

Complex formulas and expressions can be postulated in axioms and param- 
eterized by means of logic functions. Functions are a means of introducing 
encapsulation and genericity for formulas and complex expressions. For exam- 
ple, the previous formula can be parameterized into function is-max with the 
following signature and definition: 

function is-max (m: int. A: array int, low: int, high: int) 
returns ( bool ) 

{ V j : int • low < j A j < high A[j] < m} 

Axioms constrain global constants, variables, and functions; they are useful 
to supply Boogie with domain knowledge to facilitate inference and guide the 
automated reasoning over non-trivial programs. In certain situations it might 
for example be helpful to introduce the property that if an array has maximum 
m over range [low. .high] and the element in position high + 1 is smaller than m 
then m is also the maximum over range [low. .high + 1]. The following Boogie 
axiom will express this: 

axiom ( Vm: int. A: array int, low: int, high: int • 
is-max{m, A, low, high) A A[high -\- 1] <m 
=^ is-vnax {m. A, low, high+1) ) 

3.2.2 The Boogie programming language 

A Boogie program is a collection of procedures. Each procedure consists of 
a signature, a specification and (optionally) an implementation or body. The 
signature gives the procedure a name and declares its formal input and output 
arguments. The specification is a collection of contract clauses of three types: 
frame conditions, preconditions, and postconditions. 

A frame condition, introduced by keyword modifies, consists of a list of 
global variables that can be modified by the procedure; it is useful in evaluating 
the side-effects of procedure call within any context. A precondition, introduced 
by the keyword requires, is a formula that is required to hold upon procedure 
invocation. A postcondition, introduced by the keyword ensures, is a formula 
that is guaranteed to hold upon successful termination of the procedure. For 
example, procedure max_v2, computing the maximum value in an array A given 
its size n, has the following specification. 

procedure max_v2 (A: array int, n: int) returns {m: int) 
requires n > 1; 
ensures is_max (m. A, 1, n); 

The implementation of a procedure consists of a declaration of local vari- 
ables, followed by a sequence of (possibly labeled) program statements. Figure 
[T] shows a simplified syntax for Boogie statements. Statements of class Anno- 
tation introduce checks at any program point: an assertion is a formula that 
must hold of every execution that reaches it for the program to be correct and 
an assumption is a formula whose validity at the program point is postulated. 
Statements of class Modification affect the value of program variables, by nonde- 
terministically drawing a value for them (havoc), assigning them the value of an 
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Statement 

Annotation 
Modification 

ConditionalBranch 
Loop 
Invariant 



Assertion | Modification 

ConditionalBranch | Loop 

assert Formula | assume Formula 

havoc Variablcid | Variablcid :— Expression 

call [ Variablcid"^ :— ] Proccdurcid ( Expression* 

if ( Formula ) Statement* [ else Statement* ] 

while ( Formula ) Invariant* Statement* 

invariant Formula 



Figure 1: Simplified abstract syntax of Boogie statements 



expression (:=), or calling a procedure with actual arguments (call). The usual 
conditional if statement controls the execution flow. Finally, the while state- 
ment supports loop iteration, where any loop can be optionally annotated with 
a number of Invariants (see Section [3. ip . Boogie can check whether Definition 
[T] holds for any user-provided loop invariant. 
The implementation of procedure max-v2 is: 

var i: int; 

z — 1; m A[l]] 
while {i< n) 

{ 

if {m<A[i\) { m := A[i\- } 
i := i + 1; 

} 

While the fuU Boogie language includes more types of statement, any Boogie 
statement can be desugared into one of those in Figure[T] In particular, the only 
looping construct we consider is the structured while; this choice simplifies the 
presentation of our loop invariant inference technique and makes it closer as 
if it was defined directly on a mainstream high-level programming language. 
Also, there is a direct correspondence between Boogie's while loop and Eiffel's 
from ... until loop, used in the examples of Section [2] and the definitions in 
Section 13.11 

3.3 Notational conventions 

subExp((/), SubType) denotes the set of sub-expressions of formula (f> that are of 
syntactic type SubType. For example, subExp(is-max{v,X,l,n), Map) denotes 
all mapping sub-expressions in is-max{v,X,l,n), that is only X[j] . 

replace((/), oW, new, *) denotes the formula obtained from by replacing ev- 
ery occurrence of sub-expression old by expression new. Similarly, replace(0, old, 
new, n) denotes the formula obtained from (j) by replacing only the n-th oc- 
currence of sub-expression old by expression new, where the total ordering of 
sub-expressions is given by a pre-order traversal of the expression parse tree. 
For example, rep\ace{is-max{v,X,l,n),j,h,'i') is: 

V h : int • low< h A h< high => A[h] < m 
while rep\ace{is_max{v,X,l,n),j,h,4:) is: 

V j : int • low< j A j < high => A[h] < m 
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1 findJnvariants ( a_procedure: PROCEDURE ) 



2 : SET.OF [FORMULA] 

3 do 

4 Result := 

5 for each post in postconditions( a_procedwre) do 

6 for each loop in outer Joops{a-procedure) do 

7 compute all weakenings of post 

8 according to chosen strategies 

9 weakenings := build_weakenings(post, loop) 

10 for each formula in weakenings do 

11 for each anyJoop in \oops{a_procedure) do 

12 if is_invariant(/ormM^a, anyJoop) then 

13 Result := Result U {formula} 



Figure 2: Procedure findJnvariants 

Given a while loop while ( ... ) { iJody }, targets(f) denotes the set of its 
targets: variables (including mappings) that can be modified by its Body; this 
includes global variables that appear in the modifies clause of called procedures. 

Given a procedure foo, variables(/oo) denotes the set of all variables that 
are visible within foo, that is its locals and any global variable. 

A loop i' is nested within another loop £, and we write £' -< £, iff £' belongs 
to the Body of £. Notice that ii £' ^ £ then targets(^') C targets(£). Given a 
procedure foo, its outer while loops are those in its body that are not nested 
within any other loop. 

4 Generating loop invariants from postconditions 

This section presents the loop invariant generation algorithm in some detail. 
4.1 Main algorithm 

The pseudocode in Figure [5] describes the main algorithm for loop-invariant 
generation. The algorithm operates on a given procedure and returns a set of 
formulas that are invariant of some loop in the procedure. Every postcondition 
post among all postconditions postconditions(a43roce(iMre) of the procedure is 
considered separately (line 5). This is a coarse-grained yet effective way of im- 
plementing the term-dropping strategy outlined in Section 12.41 the syntax of 
the specification language supports splitting postconditions into a number of 
conjuncts, each introduced by the ensures keyword, hence each of these con- 
juncts is weakened in isolation. It is reasonable to assume that the splitting into 
ensures clauses performed by the user separates logically separated portions 
of the postcondition, hence it makes sense to analyze each of them separately. 
This assumption might fail, of course, and in such cases the algorithm can be 
enhanced to consider more complex combinations of portions of the postcondi- 
tion. However, one should only move to this more complex analysis if the basic 
strategy — which is often effective — fails. This enhancement belongs to future 
work. 
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1 build.weakenings ( post: FORMULA; loop: LOOP ) 



2 : SET_OF [FORMULA] 

3 do 

4 Result :— {post} 

5 all-Subexpressions subExp(post, /d) U 

6 s\sbExp{post, N umber) \J 

7 subExp(post, Afap) 

8 for each constant in alLsubexpressions\target5{loop) do 

9 for each variable in targets(^oop) do 

10 Result := Result U 

11 coupled_weakenings(pos<, constant, variable) U 

12 uncoupled_weakenings(post, constant, variable) 



Figure 3: Procedure build_weakenings 

The algorithm of Figure [3] then considers every outer while loop (line 6). 
For each of them, it computes a set of weakenings of postcondition post (line 
9) according to the heuristics of Section [21 It then examines each weakening to 
determine if it is invariant to any loop in the procedure under analysis (lines 
10-13), and finally it returns the set Result of weakened postconditions that 
are invariants to some loop. For is jr\yar\ant{ formula, loop), the check consists 
of verifying whether initiation and consecution hold for formula with respect to 
loop, according to Definition[TJ How to do this in practice is non-trivial, because 
loop invariants of different loops within the same procedure may interact in a 
circular way: the validity of one of them can be established only if the validity 
of the others is known already and vice versa. 

This was the case of partition presented in Section [^751 Establishing conse- 
cution for the weakened postcondition in the outer loop requires knowing that 
the same weakened postcondition is invariant to each of the two internal while 
loops (lines 8-13) because they belong to the body of the outer while loop. At 
the same time, establishing initiation for the first internal loop requires that 
consecution holds for the outer while loop, as every new iteration of the exter- 
nal loop initializes the first internal loop. Section [5] discusses a straightforward, 
yet effective, solution to this problem. 

4.2 Building weakened postconditions 

Algorithm build_weakenings(post, /oop), described in Figure |31 computes a set 
of weakened versions of postcondition formula post with respect to outer while 
loop loop. It first includes the unchanged postcondition among the weakenings 
(line 4). Then, it computes (lines 5-7) a list of sub-expressions of post made 
of atomic variable identifiers (syntactic class Id), numeric constants (syntac- 
tic class Number) and references to elements in arrays (syntactic class Map). 
Each of these sub-expressions that loop does not modify (i.e., it is not one of 
its targets) is a constant with respect to the loop. The algorithm then ap- 
plies the constant relaxation heuristics of Section 12.11 by relaxing constant into 
any variable among the loop^s targets (lines 8-9). More precisely, it computes 
two sets of weakenings for each pair {constant, variable): in one uncoupling. 
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1 coupled-weakenings 

2 ( post: FORMULA; constant, variable: EXPRESSION ) 

3 : SET.OF [FORMULA] 

4 do 

5 Result := rep\ace{post, constant, variable,*) 

6 aged_variable := digmg{v ariable, loop) 

7 Result Result U 

8 replace(posi, constant, aged-variable, *) 

Figure 4: Procedure coupled_weakenings 

described in Section [^31 is also applied (lines 12 and 11, respectively). 

The fact that any target of the loop is a candidate for substitution justifies 
our choice of considering only outer while loops: if a loop £' is nested within 
another loop t then targets(^') C targets(£), so considering outer while loops is 
a conservative approximation that does not overlook any possible substitution. 

4.3 Coupled weakenings 

The algorithm in Figure 0] applies the constant relaxation heuristics to postcon- 
dition post without uncoupling. Hence, relaxing constant into variable simply 
amounts to replacing every occurrence of constant by variable in post (line 
5); i.e., rep\ace{post, constant, variable, *) using the notation introduced in Sec- 
tion l3.3l Afterward, the algorithm applies the other aging heuristics (introduced 
in Section 12.21) : it computes the "previous" value of variable in an execution 
of loop (line 6) and it substitutes the resulting expression for constant in post 
(lines 7-8). 

While the implementation of function aging could be very complex we adopt 
the following unsophisticated approach. For every possible acyclic execution 
path in loop, we compute the symbolic value of variable with initial value vq 
as a symbolic expression e(t;o)- Then we obtain ag\ng(variable,loop) by solving 
the equation e(t;o) — variable for vq, for every execution patho For example, if 
the loop simply increments variable by one, then e(wo) = + 1 and therefore 
ag\ng(variable,loop) = variable — 1. Again, while the example is unsophisti- 
cated it is quite effective in practice; indeed, most of the times it is enough to 
consider simple increments or decrements of variable to get a "good enough" 
aged expression. 

4.4 Uncoupled weakenings 

The algorithm of Figure [5] is a variation of the algorithm of Figure 2] applying the 
uncoupling heuristics outlined in Section [2?3l It achieves this by considering ev- 
ery occurrence of constant in post separately when performing the substitution 
of constant into variable (line 6). Everything else is as in the non- uncoupled 
case; in particular, aging is applied to every candidate for substitution. 

^Note that 3g\r{g{variable, loop) is in general a set of expressions, so the notation at lines 
6—8 in Figure |4] is a convenient shorthand. 
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1 uncoupled-weakenings 

2 {post: FORMULA; constant, variable: EXPRESSION) 

3 : SET.OF [FORMULA] 

4 do 

5 Result := 0; index := 1 

6 for each occurrence of constant in post do 

7 Result := Result U 

8 {replace(post, constant, variable, index)} 

9 aged_variable zgmg{v ariable, loop) 

10 Result := Result U 

11 {replace(post, constant, agedjvariable, index)} 

12 index := index + 1 

Figure 5: Procedure uncoupled_weakenings 

This implementation of uncoupling relaxes one occurrence of a constant at 
a time. This is not the most general implementation of uncoupling, as in some 
cases it might be useful to substitute different occurrences of the same constant 
by different variables. This was the case of partition discussed in Section 12.31 
where relaxing two occurrences of the same constant Result into two differ- 
ent variables was needed in order to get a valid invariant. Section 12.41 showed, 
however, that the term-dropping heuristics would have made this "double" re- 
laxation unnecessary for the procedure. 

5 Implementation and experiments 

We developed a command-line tool code-named gin-pink (Generation of INvari- 
ants by PostcondltioN weaKening) implementing in Eiffel the loop-invariant 
inference technique described in Section 2] While we plan to integrate gin-pink 
into EVE (the Eiffel Verification Environment!^) where it will analyze the code 
resulting from the translation of Eiffel into Boogie IJO] , its availability as a stand- 
alone tool makes it possible to use it for languages other than Eiffel provided a 
Boogie translator is available. 

gin-pink applies the algorithm of Figure [5] to some selected procedure in a 
Boogie file provided by the user. After generating all weakened postconditions 
it invokes the Boogie tool to determine which of them is indeed an invariant: for 
every candidate invariant /, a new copy of the original Boogie file is generated 
with / declared as invariant of all loops in the procedure under analysis. It then 
repeats the following until either Boogie has verified all current declarations of 
/ in the file or no more instances of / exist in the procedure: 

1. Use Boogie to check whether the current instances of / are verified invari- 
ants, that is they satisfy initiation and consecution. 

2. If any candidate fails this check, comment it out of the file. 

In the end, the file contains all invariants that survive the check, as well as a 
number of invariants that could not be checked in the form of comments. If 

^^http : //eve . origo . ethz . ch 
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no invariant survives or the verified invariants are unsatisfactory, the user can 
still manually inspect the generated files to see if verification failed due to the 
limited reasoning capabilities of Boogie. 

When generating candidate invariants, gin-pink does not apply all heuris- 
tics at once but it tries them incrementally, according to user-supplied options. 
Typically, the user starts out with just constant relaxation and checks if some 
non-trivial invariant is found. If not, the analysis is refined by gradually in- 
troducing the other heuristics — and thus increasing the number of candidate 
invariants as well. In the experiments below we briefly discuss how often and 
to what extent this is necessary in practice. 

Experiments Table [T] summarizes the results of a number of experiments 
with gin-pink with a number of Boogie procedures obtained from Eiffel code. 
We carried out the experimental evaluation as follows. First, we collected ex- 
amples from various sources [221 [IHl 1331 HZl [3] and we manually completed the 
annotations of every algorithm with full pre and postconditions as well as with 
any loop invariant or intermediate assertion needed in the correctness proof. 
Then, we coded and tried to verify the annotated programs in Boogie, sup- 
plying some background theory to support the reasoning whenever necessary. 
The latest Boogie technology cannot verify certain classes of properties with- 
out a very sophisticated ad hoc background theory or without abstracting away 
certain aspects of the implementation under verification. For example, in our 
implementation of Bubblesort, Boogie had difficulties proving that the output 
is a permutation of the input. Correspondingly, we omitted the (few) parts of 
the specification that Boogie could not prove even with a detailedly annotated 
program. Indeed, "completeness" (full functional correctness) should not be a 
primary concern, because its significance depends on properties of the prover 
(here Boogie), orthogonal to the task of inferring invariants. Finally, we ran 
gin-pink on each of the examples after commenting out all annotations except 
for pre and postconditions (but leaving the simple background theories in); in a 
few difficult cases (discussed next) we ran additional experiments with some of 
the annotations left in. After running the tests, we measured the relevance of 
every automatically inferred invariant: we call an inferred invariant relevant if 
the correctness proof needs it. Notice that our choice of omitting postcondition 
clauses that Boogie cannot prove does not infiuence relevance, which only mea- 
sures the fraction of inferred invariants that are useful for proving correctness. 

For each experiment. Table [T] reports: the name of the procedure under 
analysis; the length in lines of codes (the whole file including annotations and 
auxiliary procedures and, in parentheses, just the main procedure); the total 
number of loops (and the maximum number of nested loops, in parentheses); the 
total number of variables modified by the loops (scalar variables/ array or map 
variables); the number of weakened postconditions (i.e., candidate invariants) 
generated by the tool; how many invariants it finds; the number and percentage 
of verified invariants that are relevant; the total run-time of gin-pink in seconds; 
the source (if any) of the implementation and the annotations. The experiments 
where performed on a PC equipped with an Intel Quad-Core 2.40 GHz CPU and 
4 Gb of RAM, running Windows XP as guest operating system on a VirtualBox 
virtual machine hosted by Ubuntu GNU/Linux 9.04 with kernel 2.6.28. 

Most of the experiments already succeeded with the application of the most 



15 



Procedure 


LOG 


# LP- 


M.V. 


OND. 


INV. 


REL. 


T. 


Src. 


Array Partitioning (vl) 


58 


(22) 


1 (1) 


2/1 


38 


9 


3 ( 33%) 


93 




Array Partitioning (v2) 


68 


(40) 


3(2) 


2/1 


45 


2 


2 (100%) 


205 


[29l 


Array Stack Reversal 


147 


(34) 


2 (1) 


1/2 


134 


4 


2 ( 50%) 


529 




Array Stack Reversal (ann.) 


147 


(34) 


2 (1) 


1/2 


134 


6 


4 ( bi/o) 


516 




Bubblesort 


69 


(29) 


2(2) 


2/1 


14 


2 


2 (100%) 


65 


[33] 


Coincidence Count 


59 


(29) 


1 (1) 


3/0 


1351 


1 


1 (100%) 


4304 


[27] 


Dutch National Flag 


77 


(43) 


1 (1) 


3/1 


42 


10 


2 ( 20%) 


117 


[H] 


Dutch National Flag (ann.) 


77 


(43) 


1 (1) 


3/1 


42 


12 


4 ( 33%) 


122 


[16] 


Majority Count 


48 


(37) 


1 (1) 


3/0 


23 


5 


2 (40%) 


62 


mi32l 


Max of Array (vl) 


27 


(17) 


1 (1) 


2/0 


13 


1 


1 (100%) 


30 




Max of Array (v2) 


27 


(17) 


1 (1) 


2/0 


7 


1 


1 (100%) 


16 




Plateau 


53 


(29) 


1 (1) 


3/0 


31 


6 


3 ( 50%) 


666 


[H 


Sequential Search (vl) 


34 


(26) 


1 (1) 


3/0 


45 


9 


5 ( 56%) 


120 




Sequential Search (v2) 


29 


(21) 


1 (1) 


3/0 


24 


6 


6 (100%) 


58 




Shortest Path (ann.) 


57 


(44) 


1 (1) 


1/4 


23 


2 


2 (100%) 


53 


m 


Stack Search 


196 


(49) 


2 (1) 


1/3 


102 


3 


3 (100%) 


300 




Sum of Array 


26 


(15) 


1 (1) 


2/0 


13 


1 


1 (100%) 


44 




Topological Sort (ann.) 


65 


(48) 


1 (1) 


2/4 


21 


3 


2 (67%) 


101 


[31] 


Welfare Crook 


53 


(21) 


1 (1) 


3/0 


20 


2 


2 (100%) 


586 


[191 



Table 1: Experiments with gin-pink. 

basic weakening techniques. Procedure Coincidence Count is the only case that 
required a more sophisticated uncoupling strategy where two occurrences of the 
same constant within the same formula were weakened to two different aged 
variables. This resulted in an explosion of the number of candidate invariants 
and consequently in an experiment running for over an hour. 

A few programs raised another difficulty, due to Boogie's need for user- 
supplied loop invariants to help automated deduction. Boogie cannot verify any 
invariant in Shortest Path or Topological Sort without additional invariants ob- 
tained by means other than the application of the algorithm itself. On the other 
hand, the performance with programs Array Stack Reversal and Dutch National 
Flag improves considerably if user-supplied loop invariants are included, but fair 
results can be obtained even without any such annotation. Table [T] reports both 
experiments, with and without user-supplied annotations. 

More generally, Boogie's reasoning abilities are limited by the amount of 
information provided in the input file in the form of axioms and functions that 
postulate sound inference rules for the program at hand. We tried to limit 
this amount as much as possible by developing the necessary theories before 
tackling invariant generation. In other words, the axiomatizations provided are 
enough for Boogie to prove functional correctness with a properly annotated 
program, but we did not strengthen them only to ameliorate the inference of 
invariants. We believe that a richer axiomatization could have removed the need 
for user-supplied invariants in the programs considered. 

6 Discussion and related work 
6.1 Discussion 

The experiments in Section [5] provide a partial, yet significant, assessment of 
the practicality and effectiveness of our technique for loop invariant inference. 
Two important factors to evaluate any inference technique deserve comment: 
relevance of the inferred invariants and scalability to larger programs. 

A large portion of the invariants retrieved by gin-pink are relevant — i.e.. 
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required for a functional correctness proof — and complex — i.e., involving 
first-order quantification over several program elements. To some extent, this 
is unsurprising because deriving invariants from postconditions ensures by con- 
struction that they play a central role in the correctness proof and that they 
are at least as complex as the postcondition. 

As for scalability to larger programs, the main problem is the combinatorial 
explosion of the candidate invariants to be checked as the number of variables 
that are modified by the loop increases. In properly engineered code, each 
routine should not be too large or call too many other routines. The empirical 
observations mentioned in [24l Sec. 9] seem to support this assumption, which 
ensures that the candidate invariants do not proliferate and hence the inference 
technique can scale within reasonable limits. The examples of Section [5] are 
not trivial in terms of length and complexity of loops and procedures, if the 
yardstick is well-modularized code. On the other hand, there is plenty of room 
for finessing the application order of the various weakening heuristics in order 
to analyze the most "promising" candidates first; the Houdini approach [17] 
might also be useful in this context. The investigation of these aspects belongs 
to future work. 

6.2 Limitations 

Relevant invariants obtained by postcondition weakening are most of the times 
significant, practically useful, and complementary to a large extent to the cate- 
gories that are better tackled by other methods (see next sub-section). Still, the 
postcondition weakening technique cannot obtain every relevant invariant. Fail- 
ures have two main different origins: conceptual limitations and shortcomings 
of the currently used technology. 

The first category covers invariants that are not expressible as weakening of 
the postcondition. This is the case, in particular, whenever an invariant refers 
to a local variable whose final state is not mentioned in the postcondition. For 
example, the postcondition of procedure max in Section 12.11 does not mention 
variable i because its final value n is not relevant for the correctness. Corre- 
spondingly, invariant i < n — which is involved in the partial correctness proof 
— cannot be obtained by postcondition weakening. A potential solution to 
these conceptual limitations is two-fold: on the one hand, many of these invari- 
ants that escape postcondition weakening can be obtained reliably with other 
inference techniques that do not require postconditions — this is the case of 
invariant i < n in procedure max which is retrieved automatically by Boogie. 
On the other hand, if we can augment postconditions with complete information 
about local variables, the weakening approach can have a chance to work. In 
the case of max^ a dynamic technique could suggest the supplementary post- 
condition i < n A i >n which would give the sought invariant by dropping the 
second conjunct. 

Shortcomings of the second category follow from limitations of state-of-the- 
art automated theorem provers, which prevent reasoning about certain inter- 
esting classes of algorithms. As a simple example, consider the following im- 
plementation of Newton's algorithm for the square root of a real number, more 
precisely the variant known as the Babylonian algorithm (we ignore numerical 
precision issues) [5S]: 
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square-root (a: REAL): REAL 
require a > 
local y: REAL 
do 

from Result := 1; y := a 
until Result — y 
loop 

Result := (Result + y)/2 
y := a / Result 
end 

ensure Result > A Result * Result = a 

Postcondition weakening would correctly find invariant Result * y = a (by term 
dropping and uncoupling), but Boogie cannot verify that it is an invariant be- 
cause the embedded theorem prover Z3 does not handle reasoning about prop- 
erties of products of numeric variables |27j . If we can verify by other means 
that a candidate is indeed an invariant, the postcondition weakening technique 
of this paper would be effective over additional classes of programs. 

6.3 Related work 

The amount of research work on the automated inference of invariants is formi- 
dable and spread over more than three decades; this reflects the cardinal role 
that invariants play in the formal analysis and verification of programs. This 
section outlines a few fundamental approaches and provides some evidence that 
this paper's technique is complementary, in terms of kinds of invariants inferred, 
to previously published approaches. For more references, in particular regarding 
software engineering applications, see the "related work" section of [15j. 

Static methods Historically, the earliest methods for invariant inference where 
static as in the pioneering work of Karr |23j . Abstract interpretation and the 
constraint-based approach are the two most widespread frameworks for static 
invariant inference (see also [51 Chap. 12]). 

Abstract interpretation is, roughly, a symbolic execution of programs over 
abstract domains that over-approximates the semantics of loop iteration. Since 
the seminal work by Cousot and Cousot [TU], the technique has been updated 
and extended to deal with features of modern programming languages such as 
object-orientation and heap memory-management (e.g., pSl 

Constraint-based techniques rely on sophisticated decision procedures over 
non-trivial mathematical domains (such as polynomials or convex polyhedra) 
to represent concisely the semantics of loops with respect to certain template 
properties. 

Static methods are sound — as is the technique introduced in this paper 
— and often complete with respect to the class of invariants that they can 
infer. Soundness and completeness are achieved by leveraging the decidabil- 
ity of the underlying mathematical domains they represent; this implies that 
the extension of these techniques to new classes of properties is often limited 
by undecidability. In fact, state-of-the-art static techniques can mostly infer 
invariants in the form of "well-behaving" mathematical domains such as linear 
inequalities [TTJ [5] , polynomials [351 i restricted properties of arrays [3 [51 dD] , 
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and linear arithmetic with uninterpreted functions [T] . Loop invariants in these 
forms are extremely useful but rarely sufficient to prove full functional correct- 
ness of programs. In fact, one of the main successes of abstract interpretation 
has been the development of sound but incomplete tools [5] that can verify the 
absence of simple and common programming errors such as division by zero or 
void dereferencing. Static techniques for invariant inference are now routinely 
part of modern static checkers such as ESC/ Java [TB], Boogie/Spec# [35], and 
Why/Krakatoa/Caduceus [32]. 

The technique of the present paper is complementary to most static tech- 
niques in terms of the kinds of invariant that it can infer, because it derives 
invariants directly from postconditions. In this respect "classic" static inference 
and our inference by means of postcondition weakening can fruitfully work to- 
gether to facilitate functional verification; to some extent this happens already 
when complementing Boogie's built-in facilities for invariant inference with our 
own technique. 

[34||2T1 [IS [25l [24] are the approaches that, for different reasons, share more 
similarities with ours. To our knowledge, [Ml [JT] [T31 [15] are the only other works 
applying a static approach to derive loop invariants from annotations. [21] re- 
lies on user-provided assertions nested within loop bodies and essentially tries 
to check whether they hold as invariants of the loop. This does not release the 
burden of writing annotations nested within the code, which is quite complex 
as opposed to providing only contracts in the form of pre and postconditions. 
In practice, the method of [21| works only when the user-provided annotations 
are very close to the actual invariant; in fact the few examples where the tech- 
nique works are quite simple and the resulting invariants are usually obtainable 
by other techniques that do not need annotations. [13] briefly discusses deriv- 
ing the invariant of a for loop from its postcondition, within a framework for 
reasoning about programs written in a specialized programming language. [25] 
also leverages specifications to derive intermediate assertions, but focusing on 
lower-level and type-like properties of pointers. On the other hand, [33] derives 
candidate invariants from postconditions in a very different setting than ours, 
with symbolic execution and model-checking techniques. 

Finally, [21] derives complex loop invariants by first encoding the loop se- 
mantics as recurring relations and then instructing a rewrite-based theorem 
prover to try to remove the dependency on the iterator variable(s) in the rela- 
tions. It shares with our work a practical attitude that favors powerful heuristics 
over completeness and leverages state-of-the-art verification tools to boost the 
inference of additional annotations. 

Dynamic methods More recently, dynamic techniques have been applied 
to invariant inference. The Daikon approach of Ernst et al. [TS] showed that 
dynamic inference is practical and sprung much derivative work (e.g., |35[|12[|5B] 
and many others) . In a nutshell, the Daikon approach consists in testing a large 
number of candidate properties against several program runs; the properties 
that are not violated in any of the runs are retained as "likely" invariants. 
This implies that the inference is not sound but only an "educated guess" : 
dynamic invariant inference is to static inference what testing is to program 
proofs. Nonetheless, just like testing is quite effective and useful in practice, 
dynamic invariant inference is efficacious and many of the guessed invariants 
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are indeed sound. 

Our approach shares with the Daikon approach the idea of guessing a candi- 
date invariant and testing it a posteriori. There is an obvious difference between 
our approach, which retains only invariants that can be soundly verified, and 
dynamic inference techniques, which rely on a finite set of tests. A deeper differ- 
ence is that Daikon guesses candidate invariants almost blindly, by trying out a 
pre-defined set of user-provided templates (including comparisons between vari- 
ables, simple inequalities, and simple list comprehensions). On the contrary, our 
technique assumes the availability of contracts (and postconditions in particular) 
and leverages it to restrict quickly the state-space of search and get to good- 
quality loop invariants in a short time. As it is the case for static techniques, 
dynamic invariant inference methods can also be usefully combined with our 
technique, in such a way that invariants discovered by dynamic methods boost 
the application of the postcondition-weakening approach. 

Program construction Classical formal methods for program construction 
[m [T9j [29l [32] have first described the idea of deriving loop invariants from post- 
conditions. Several of the heuristics that we discussed in Section [2] are indeed 
a rigorous and detailed rendition of some ideas informally presented in [TH] . 
In addition, the focus of the seminal work on program construction is to derive 
systematically an implementation from a complete functional specification. In 
this paper the goal is instead to enrich the assertions of an already implemented 
program and to exploit its contracts to annotate the code with useful invariants 
that facilitate a functional correctness proof. 

7 Conclusion and future work 

As we hope to have shown, taking advantage of postconditions makes it possible 
to obtain loop invariants through effective techniques — not as predictable as 
the algorithms that yield verification conditions for basic constructs such as 
assignments and conditionals, but sufficiently straightforward to be applied by 
tools, and yielding satisfactory results in many practical cases. 

The method appears general enough, covering most cases in which a pro- 
grammer with a strong background in Hoare logic would be able at some effort 
to derive the invariant, but a less experienced one would be befuddled. So it 
does appear to fill what may be the biggest practical obstacle to automatic 
program proving. 

The method requires that the programmer (or a different person, the "proof 
engineer" , complementing the programmer's work, as testers traditionally do) 
provide the postcondition for every routine. As has been discussed in Section [TJ 
we feel that this is a reasonable expectation for serious development, refiected 
in the Design by Contract methodology. For some people, however, the very 
idea of asking programmers or other members of a development team to come 
up with contracts of any kind is unacceptable. With such an a priori assump- 
tion, the results of this paper will be of little practical value; the only hope is 
to rely on invariant inference techniques that require the program only (com- 
plemented, in approaches such as Daikon, by test results and a repertoire of 
invariant patterns). 
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Some of the results that the present approach yields (sometimes trivially) 
when it is applied manually, are not yet available through the tools used in the 
current implementation of gin-pink. Although undecidability results indicate 
that program proving will never succeed in all possible cases, it is fair to expect 
that many of these limitations — such as those following from Z3's current 
inability to handle properties of products of variables — will go away as proof 
technology continues to progress. 

We believe that the results reported here can play a significant role in the 
effort to make program proving painless and even matter-of-course. So in ad- 
dition to the obvious extensions — making sure the method covers all effective 
patterns of postcondition weakening, and taking advantage of progress in the- 
orem prover technology — our most important task for the near future is to 
integrate the results of this article, as unobtrusively as possible for the practic- 
ing programmer, in the background of a verification environment for contracted 
object-oriented software components. 
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